Papers with scalable preference data construction strategy
Finding the Sweet Spot: Preference Data Construction for Scaling Preference Optimization (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models generate unintended outputs due to their unsupervised nature. |
| Approach: | They propose a method to construct preference pairs of selected and rejected LLMs by repeated random sampling to improve alignment performance. |
| Outcome: | The proposed method improves performance as the sample size increases. |